I think it would be better for a client that supports this mechanism to hardcode the merkle root instead of the straight serialized hash, and to drop serialized_hash from this message.

Hm, yeah, I think that makes sense, I was struggling with finding the right path between building on top of the existing assumeutxo data we already have and extending it. I am adding the merkle root to the assumeutxo data in my PR, so checking the serialized_hash is a belts and suspenders there, so it makes sense to make this explicit here as well.

If you wanted to change the contents of the utxo dump as well, it would be nice to include the header chain for the block where the snapshot was taken. Then you could do an import without needing to connect to the network at all, I think.

I think it would be better for a client that supports this mechanism to hardcode the merkle root instead of the straight serialized hash, and to drop serialized_hash from this message.

Dropped serialized_hash mentions from the BIP completely in this new version.

it would be nice to include the header chain for the block where the snapshot was taken

Hm, I find this a bit odd. For the purpose of this BIP we are connected to the network necessarily, so we can just as well fetch the headers over the network in the way we usually do and that should be simpler code-wise. For loading the file directly it would make more sense to me but still we are connecting to the network afterwards. I will need to think about this more but thankfully we have the versioning of the dump file so we can extend the format easily in the future and it wouldn't even need to affect this BIP I think.

Rather than a proof, it might be better to just request getutxoset <height> <hash> 0xFFFFFFFF once to get the full list of chunk hashes -- that should be about 74kB for a 9GB utxo set, and should stay under 4MB until the utxo set is >450GB. Then each chunk is just <height> <hash> <number> <data>.

Huh, interesting idea to get all the chunk hashes first, I didn't think of that. It might make sense to do this with a separate message type even instead of hacking getutxoset as you described. I will have to think about it a little more.

Specified the fetching of the chunk hash list now with new message type getutxotree/utxotree messages. I liked this a lot more when combining it with dropping the discovery approach and asking for the specific UTXO set directly. Let me know what you think.

I think this process defeats the point of the service bit. Having "at least one" UTXO set only "works" while there are only a small number of UTXO set options to download. If it's not sufficient to just try peers until you find the UTXO set you want, then we probably need a way to advertise specific UTXO sets.

Thanks, I do think it should be sufficient to try peers until we find the UTXO set we want and I dropped the discovery approach, see my other comment as well.

I'm not sure discovery is a useful feature. It enables a misguided implementation to blindly trust whatever nodes provide, and harms node privacy by adding a way to fingerprint nodes.

If you know what you'll accept in advance, you can simply request that snapshot, and the node will either provide it or (potentially) say it can't.

I dropped the discovery approach, I think fingerprinting potential is still there with some probing but it's still better than with discovery and I think this matches the approach we follow in the network generally, like in BIP157 for example, better as well.

It also happens to sit just below the maximum P2P message size MAX_PROTOCOL_MESSAGE_LENGTH, so it may be clearer to refer to that instead of block size

Yeah, but this was a contious decision actually. MAX_PROTOCOL_MESSAGE_LENGTH is a Bitcoin Core implementation specific value. A different implementation may have a higher value for this. But every implementation will at least need to be able to receive the biggest possible block. So I think it's better to anchor it to that.

nit: I think it would be clearer to repeat here that we should disconnect a node that doesn't respond to our getutxoset, as stated above:

A node signaling NODE_UTXO_SET MUST respond to getutxosetinfo messages and MUST be capable of serving all UTXO sets it advertises in its utxosetinfo response. A node that fails to meet these
obligations SHOULD be disconnected.

With the latest changes I felt it would be more appropriate to remove this general statement instead:

A node that fails to meet these obligations SHOULD be disconnected.

Without discovery it is just harder for us to decide if the peer is misbehaving or not. It is more precise anyway to clarify this explicitly for each situation.

I'm trying to understand why we "should" disconnect here, but we "must" disconnect if the peer sends us a utxoset message that fails the proof verification (L176). Is it because in the second case, the peer is surely trying to trick us, while in the first case, we probably have a bug on our end?

I'm trying to understand how it can happen that our chunks pass the individual verifications, but the final utxo set hash mismatches, and whether in this case it would be a problem on our end, or peers trying to trick us.

Yeah, the intention was that we are stricter with a clear inconsistency in what the peer has given us vs. something that is probably a problem on our side. But have revisited the disconnection policies and I think it is a lot more consistent now. If I got everything right we must disconnect if the peer sent us something clearly inconsistent, we should disconnect if the peer sent us we didn't ask for and if evidence points to the issue being on our side I am now not saying anything about disconnecting.

This is in conflict with the Utreexo proposal which allocates

• Bit 12 to NODE_UTREEXO
• Bit 13 to NODE_UTREEXO_ARCHIVE

per the BIP183 draft.

Moved to take Bit 14, thanks for giving me the heads up. But I will most likely change this to use BIP 434 instead unless there are any late issues arising with it.

I guess this is safe because no left_child and right_child should ever have the same data, but I dimly remember that this construction had some downsides in the Merkle tree for transactions in a block. (Some node implementation accepted a block with the same transaction repeated, or smth?) Would it perhaps be better to have a dedicated value to add into the hash instead of repeating the left_child, e.g., SHA256d(left_child || SHA256d(odd_number_of_leaves_no_right_child))?

Hm, while the situation is a bit different than with txs, it's a good point to raise and one way or the other, there should be an additional check added to the BIP. Generally, I think any such attack should be able to get caught based on the utxotree response. The attack you describe would make it necessary for the attack to have the same hash in the list twice. So we should be able to catch it with a duplication check of the chunk hash list in the utxotree response. The alternative you are suggesting should probably be to use SHA256d("") instead of the duplication but I still think we can not avoid doing a similar check. If an attacker sends us the list with SHA256d("") included at the end we need to identify it early, otherwise there could be a confusion about what the correct number of chunks is. The attacker could not send the final ghost chunk to us and we would keep asking peers for it and get disconnected most likely, failing to complete the snapshot even though we have all the data already (the same would be possible for the current approach BTW). At least using an empty hash as the sentinel value would ensure that we don't append any nonsense data to the snapshot if we get a result to the ghost chunk.

I did some (re) reading on the latest preferred approaches for merkle trees and it appears that domain separation through tagged hashing is valid (as in Taproot) or just promoting the odd leaf up a level. Of the two I much prefer the latter due to ease of implementation. I have thus changed the draft to use this.

I am still undecided if this is the best approach since I see a little bit of confusion potential as people might just assume the merkle tree works the same as the one for txs. The uniqueness check seems like a small price to pay. But the new version is cleaner. Happy to reconsider this change if anyone feels strongly about this or if I missed anything worth considering here.

getutxoset feels a bit odd, when the message actually requests a chunk. Also, Cluster Mempool makes extensive use of the term "chunk", and I was wondering whether this overlap could cause confusion in the future.

Chunk is just a very generic term in computer science, it's part of the http spec as well and thus we have the term appearing several times in the libevent replacement code as well which can not be avoided. Initially, I also wasn't the happiest about it but I just couldn't fit a different term for it that felt right, especially considering how http uses the term and how that seems to match pretty well. This also explains the message naming: The message transports the UTXO set, chunks is just an aspect of the transport mechanism. At least that's how felt most comfortable with reasoning about it. I think Cluster Mempool would have had a more wide pick of fitting terminology to chose from but that ship has sailed. Happy to still consider a renaming if anyone has a good suggestion but all the alternatives I could think of didn't seem to fit well enough. I also obviously prefer shorter naming in order to not make squeezing it in 12 characters too awkward.

Should there perhaps also be an addition that the serving node must not reply if it has not previously sent a utxotree message to the requesting peer?

I also checked that the Utreexo BIPs don’t use the term "utxotree", but I worry that "utreexo" and "utxotree" are extremely similar and that will cause headaches in the future.

Should there perhaps also be an addition that the serving node must not reply if it has not previously sent a utxotree message to the requesting peer?

I think this would require that the serving node keeps track of every request which opens new DoS possibilities. So I would rather not require this.

I also checked that the Utreexo BIPs don’t use the term "utxotree", but I worry that "utreexo" and "utxotree" are extremely similar and that will cause headaches in the future.

Just like with the other naming response, overall happy to consider renaming if there is a good suggestion. I also didn't like this option too much due to the closeness to utreexo but I also couldn't find a better name. Using chunks in the msg name here was ruled out by me for the same reason I laid out above: chunks are a generic term within transport and don't have anything to do with the content of what is transported IMO. In the end both proposals deal with trees of UTXOs to some degree so its hard to differentiate. I guess if there was a unique brand for this proposal that could be used but I am not creative enough, best I got are "minishare" and "UTorrentXO" but nobody wants that :p

If a node is expected to source chunks from multiple different peers, is it really necessary to receive the entire tree description of 80 KB from each of the peers?

The spec says "The requesting node sends getutxotree for the desired block hash to one or more of these peers." so I think the answer is "no, it is not necessary to receive the entire tree description of 80 KB from each of the peers" -- you only send requests to the number of peers you want to receive response from. Any attempt to give different responses will (should) result in them not hashing back to your known merkle root, so all valid descriptions will be identical, AIUI.

I read line 133,134

Sent to request a single chunk of UTXO set data. The requesting node MUST have received a utxotree
for the corresponding UTXO set before sending this message.

as the serving node not being permitted to respond to getutxoset calls for a specific tree unless it previously sent a utxotree message to the same peer, but maybe I misinterpreted that. It seems to me that both aspects of the question should be clarified:

• Must a peer send getutxotree before being eligible to responses to getutxoset for the same tree?
• Is it necessary to retrieve the utxotree from multiple peers before requesting chunks?

Perhaps it would be better to send getutxotree to one node (repeating until you get a valid response), and then send getutxoset to any nodes that support utxo set sharing, with the response utxoset <hash> <n> <empty> indicating "i don't have that utxoset data" ? So instead of getutxotree / utxotree to establish whether a peer has the data you want, you send getutxoset / utxoset and either get an explicit nope or data you actually want?

@murchandamus I tightened the language you were not happy with, primarily in the Protocol Flow section. Please let me know if it works better for you now.

@ajtowns It's an interesting suggestion. I am not convinced so far since I don't really like overloading original message semantics (its a new protocol, we may at least do it right in the beginning). I also think the explicit nope doesn't gain us that much and at the same time moves us backwards with regards to @luke-jr 's concerns. Did you have specific thoughts on this? If we would do your suggestion then I would almost feel better about adding the -info discovery approach again. But I haven't thought about it that much, I will continue to ponder this.

nit

I looked at many other BIPs, particularly 152 and 157 in terms of the language they use for similar situations and that's how I ended up with MUST here. Do you have a specific rationale for this change? I guess you are not feeling too strongly about it (nit), so I am leaving it as is for now.